We should perhaps think of creating easier ways to just pass through openxr extensions as they are without re-encoding the data

Are you talking about body tracking or in general? I don't think if there is any good way to just passthrough data from openxr because we have to implement support for each extension individually. My plan for the next big protocol update (v21) is to add a byte vec to all stream packets, so we can add more functionality without breaking the protocol.